As is known in the art, DECface from Digital Equipment Corporation is a talking synthetic face application that is essentially a visual complement of a speech synthesizer application referred to as DECtalk, also from Digital Equipment Corporation. By combining the audio functionality of a speech synthesizer with the graphical functionality of a computer-generated face, a variety of engaging user interfaces can be developed. Examples include Internet-based Agents capable of seeking and retrieving Web documents to read, Avatars for Chat applications, and a front-end interface for kiosks.
As is known, DECface is a system that facilitates the development of applications requiring a real-time lip-synchronized synthetic face. The original version of DECface operates under Digital Unix and utilizes the X-Window system for display. The current version of the DECface now functions under W32 NT or Windows 95 operating systems and can be embedded into Web browsers as a plugin. One of the core pieces of DECface is the ability to adaptively synchronize the audio to the image being displayed as described in U.S. patent application Ser. No. 08/258,145 entitled Method and Apparatus for Producing Audio-Visual Synthetic Speech, filed by Waters et al. on Jun. 10, 1994. In Waters et al., a speech synthesizer generates fundamental speech units called phonemes, which can be converted to an audio signal. The phonemes can be translated to their visual complements called visemes (known as distinct mouth postures). The result is a sequence of facial gestures approximating the gestures of speech.
One of the components of DECface is face modeling, where a given synthetic, or real image of a face, can be mapped onto a wireframe geometry that has be shaped to the contours and features of a face. As is known in this art, the process of face modeling has been manual, labor intensive and time consuming, as every node on the wireframe geometry has to be positioned by hand. In the manual process, a new image of the face is displayed onto which a wireframe geometry is roughly positioned. The user then steps though each individual node in the face geometry and repositions the current selected two dimensional (2-D) node location to match the contour, or feature, of the face.
In the prior art, some techniques for mapping geometry to images of faces have been documented. One such exemplary method relies on automatically detecting eyes, tip of the nose, and mouth using eigenfeatures as templates. The face geometry is then initialized by affine warping the face geometric model based on the detected facial feature locations. However, this form of global transformation is generally an approximation, as facial ratios across different faces do not change linearly in general.
Another exemplary method of initializing a facial geometry is to manually mark out specific facial features such as eyebrow centers and mouth corners. In addition to the requirement of manual selection, the face model in this exemplary method requires explicit connectivity information in the computation of the new node location within the image.
It is highly desirable to accurately map the geometry of DECface to an arbitrary frontal image of a face. Furthermore, expediting the process from a fully manual process to a semi-automatic procedure can dramatically reduce the time taken to map a new image of a person's face.